Methods of Performing Error Concealment For Digital Video

ABSTRACT

Error concealment is used to hide the effects of errors detected within digital video information. A complex error concealment mode decision is disclosed to determine whether spatial error concealment (SEC) or temporal error concealment (TEC) should be used. The error concealment mode decision system uses different methods depending on whether the damaged frame is an intra-frame or an inter-frame. If the video frame is an intra-frame then a similarity metric is used to determine if the intra-frame represents a scene-change or not. If the video frame is an intra-frame, a complex multi-termed equation is used to determine whether SEC or TEC should be used. A novel spatial error concealment technique is disclosed for use when the error concealment mode decision determines that spatial error concealment should be used for reconstruction. The novel spatial error concealment technique divides a corrupt macroblock into four different regions, a corner region, a row adjacent to the corner region, a column adjacent to the corner region, and a remainder main region. Those regions are then reconstructed in that order and information from earlier reconstructed regions may be used in later reconstructed regions. Finally, a macroblock refreshment technique is disclosed for preventing error propagation from harming non-corrupt inter-blocks. Specifically, an inter-macroblock may be ‘refreshed’ using spatial error concealment if there has been significant error caused damage that may cause the inter-block to propagate the errors.

FIELD OF THE INVENTION

The present invention relates to the field of digital video decoding anddisplay. In particular the present invention discloses techniques forconcealing errors in digital video caused by damaged or lost digitalvideo information.

BACKGROUND OF THE INVENTION

There has been a wireless electronics revolution in recent years due tothe rapid advancements and adoption of many new digital wirelesscommunication protocols. For example, Code Division Multiple Access(CDMA) and Global System for Mobile communication (GSM) have greatlyincreased the popularity of digital cellular telephones, Wi-Fi localarea network (LAN) protocols (such as 802.11b, 802.11g, 802.11a, etc.)have revolutionized wireless computer networks, and Bluetooth hasprovided a very useful short range wireless digital protocol for manydifferent digital devices.

Although these new digital wireless protocols are designed to beresistant to errors, digital wireless communication is always subject toinformation loss due to various physical effects on the wirelesssignals. For example, radio interference, errors caused by multi-pathreflections, radio signal shielding, range limitations, and otherproblems can degrade transmitted wireless signals such that informationbecomes lost.

Non real-time communication applications that use digital wirelessprotocols can easily handle the loss of information due to any of thesereasons by simply requesting the retransmission of the lost information.However, real-time communication applications need to be robust enoughto handle the occasional loss or corruption of information that istransmitted across a digital wireless channel.

One specific real-time communication application is the reception andimmediate display of digital video information transmitted across awireless digital communication channel. When some information is lostduring the transmission of the digital video, there is not enough timeto request and receive a retransmission of the lost information sinceretransmission brings significant delay which is not suitable forreal-time applications such as video telephony and video streaming.Thus, when digital video information is lost, the digital video receivermust attempt to display one ore more video frames using the receivedincomplete digital video information.

Several different techniques have been developed for decoding anddisplaying digital video frames as best as possible despite the loss ofsome digital video information. These techniques developed fordisplaying incomplete digital video information in the best possiblemanner are generally referred to as Error Concealment (EC) techniquessince these techniques attempt to conceal the lack of accurate videoinformation caused by errors.

The existing techniques of digital video Error Concealment (EC) mainlyfall into the following two categories: Temporal Error Concealment (TEC)and Spatial Error Concealment (SEC). Temporal Error Concealment (TEC)exploits temporal correlation between nearby (in a time dimension) videoframes. TEC conceals errors by replacing corrupt macroblocks (MBs) withthe processed macroblocks from previously reconstructed video frames. Onthe other hand, Spatial Error Concealment (SEC) exploits spatialcorrelation within a single video frame. Spatial Error Concealmentconceals errors by predicting the erroneous pixels from neighboringpixels (or transform coefficients).

Block Replacement Error Concealment (BREC) is the simplest and the moststraightforward type of Temporal Error Concealment (TEC). Blockreplacement error concealment conceals errors by replacing erroneousmacroblocks with the co-located macroblocks from a previouslyreconstructed video frame. This simple approach to temporal errorconcealment assumes a zero motion vector for the erroneous macroblocksto be concealed. The performance may be improved by utilizing moreaccurate motion vectors rather than a zero motion vector. Motion VectorProcessing (MVP) is such an example.

A very simple Spatial Error Concealment (SEC) implementation may justreplace erroneous pixels with information from the nearest availablepixels. Improvement can be achieved by utilizing interpolation andspatial prediction. The Spatial Error Concealment (SEC) may be performedeither in the pixel domain or in the frequency domain.

In order to provide the best looking video to user when errors in thecommunication stream occur, the digital video receiver should select theoptimum error concealment system. Thus, it would be desirable to createdigital video receivers that carefully select the best error concealmentsystem and then implement the error concealment systems in manners thatdisplay the best possible video image from the incomplete digital videoinformation.

SUMMARY OF THE INVENTION

The present invention introduces methods for performing errorconcealment when errors are detected within digital video information. Anumber of different techniques are presented in order to perform veryhigh quality digital video error concealment.

A first technique disclosed is a very sophisticated method ofdetermining whether spatial error concealment (SEC) or temporal errorconcealment (TEC) should be used. The main technique is to determinewhether a scene change has occurred and when a scene change has occurredthen spatial error concealment will be used (unless there are too manyerrors to perform spatial error concealment). The error concealment modedecision system uses different methods depending on whether the damagedframe is an intra-frame or an inter-frame. If the video frame is anintra-frame then a similarity metric is used to determine if theintra-frame represents a scene-change or not. If the video frame is anintra-frame, a complex multi-termed equation is used to determinewhether SEC or TEC should be used.

A novel spatial error concealment technique is disclosed for use whenthe error concealment mode decision determines that spatial errorconcealment should be used for reconstruction. The novel spatial errorconcealment technique divides a corrupt macroblock into four differentregions, a corner region, a row adjacent to the corner region, a columnadjacent to the corner region, and a remainder main region. The cornerregion is reconstructed first using adjacent macroblocks. The column androw regions are reconstructed next by using adjacent macroblocks or thereconstructed corner region in addition to adjacent macroblocks.Finally, the remainder region is interpolated using adjacentmacroblocks, the column region, the row region, and the corner region.

Finally, a macroblock refreshment technique is disclosed for preventingerror propagation from harming non-corrupt inter-blocks. Specifically,an inter-macroblock may be ‘refreshed’ using spatial error concealmentif there has been significant error caused damage that may cause theinter-block to propagate the errors. To determine when aninter-macroblock should be refreshed, a damage calculation is performed.The damage calculation considers the number of corrupt macroblocks, theamount of motion, whether a scene change has occurred, and a number ofnon corrupt intra-macroblocks.

Other objects, features, and advantages of present invention will beapparent from the accompanying drawings and from the following detaileddescription.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects, features, and advantages of the present invention will beapparent to one skilled in the art, in view of the following detaileddescription in which:

FIG. 1 illustrates the general flow of making a scene-change based errorconcealment mode decision in a digital video receiver/decoder.

FIG. 2 illustrates an example of a digital video frame macroblock mapwherein shaded areas represent corrupted macroblocks after an error thathas been detected.

FIG. 3 illustrates an example of a digital video frame macroblock mapwherein shaded areas represent corrupted macroblocks when an entireslice is deemed to be corrupt.

FIG. 4 illustrates a revised version of the flow chart of FIG. 1 whereinan improved scene-change based error concealment system is used.

FIG. 5 graphically illustrates a macroblock segmented into differentareas that are handled individually in an improved spatial errorconcealment method.

FIG. 6A illustrates the three closest neighboring pixels α, β and γ toarea F in a macroblock.

FIG. 6B illustrates a pixel map for a median adaptive predictor.

FIG. 7 illustrates the bottom-rightmost pixel of the neighboringmacroblocks that are used in an improved spatial error concealmentmethod.

FIGS. 8A, 8B, and 8C illustrate how an improved spatial errorconcealment method reconstructs a bottom row of pixels in a corruptmacroblock.

FIGS. 9A and 9B illustrate how an improved spatial error concealmentmethod reconstructs a central area of pixels in a corrupt macroblock.

DETAILED DESCRIPTION

Methods of selecting and implementing error concealment systems fordigital video are disclosed. In the following description, for purposesof explanation, specific nomenclature is set forth to provide a thoroughunderstanding of the present invention. However, it will be apparent toone skilled in the art that these specific details are not required inorder to practice the present invention. Furthermore, although thepresent invention is mainly described with reference to the MPEG digitalvideo encoding protocols, the same techniques can easily be applied toother types of digital video encoding protocols.

Error Concealment Mode Selection Overview

As set forth in the background, a digital video receiver/decoder thatreceives a disrupted digital video stream with incomplete digital videoinformation must display a video image as best it can without the lostinformation. The digital video receiver/decoder cannot request theretransmission of lost information since retransmission bringssignificant delay which is not suitable for real-time applications suchas video telephony and video streaming. Thus, several techniques havebeen developed for displaying digital video information as best aspossible despite the loss or corruption of some of the digital videoinformation. These techniques for displaying incomplete digital videoinformation in the best possible manner are generally referred to asError Concealment (EC) techniques.

Various different error concealment techniques exist for reconstructingand displaying digital video frames damaged during transmission. The twomain categories of error concealment are Temporal Error Concealment(TEC) and Spatial Error Concealment (SEC). Temporal Error Concealment(TEC) exploits temporal correlation between nearby (in a time dimension)digital video frames. Temporal error concealment often conceals errorsby replacing corrupted macroblocks (MBs) with processed macroblocks frompreviously reconstructed video frames. Spatial Error Concealment (SEC)exploits spatial correlation within a single video frame. Spatial errorconcealment conceals digital video frame errors by predicting theerroneous pixels from neighboring pixels (or transform coefficients).

Each different error concealment technique has its own advantages anddisadvantages. Thus, whether temporal error concealment or spatial errorconcealment will provide the best results will depend on the currentcircumstances. The determination of whether to use temporal errorconcealment (TEC) or spatial error concealment (SEC) is known as theError Concealment (EC) mode decision. The optimal solution to the errorconcealment mode decision is to select the error concealment method thatminimizes a given distortion metric. For example,

${Do}\left\{ \begin{matrix}{{{SE}\; C},} & {{{if}\mspace{14mu} D_{TEC}} < D_{S\; {EC}}} \\{{TEC},} & {{{if}\mspace{14mu} D_{S\; {EC}}} < D_{TEC}}\end{matrix} \right.$

wherein D_(SEC) is the distortion between SEC-concealed frames andreference frames and D_(TEC) is the distortion between TEC-concealedframes and reference frames.

However, in practice, it is very difficult to measure the distortionbecause a reference is generally not available. Thus, other methods mustbe used to determine whether a digital video receiver/decoder should usespatial error concealment or temporal error concealment.

Error Concealment Mode Based On Scene-Change

A typical video sequence consists of a significant percentage of highlytemporally correlated video frames and periodic “scene-change” videoframes that are not well temporally correlated to earlier video frames.Thus, since most video frames are highly temporally correlated, temporalerror concealment systems will outperform spatial error concealmentsystems in most circumstances. However, for a scene-change video framewherein the video frame contains a new scene that is not well correlatedto earlier digital video frames, temporal based error concealmentsystems generally provide unsatisfactory results since the similaritybetween video frames is low. Thus, spatial error concealment isgenerally a better choice for error concealment when a scene-changeoccurs.

FIG. 1 illustrates the general flow of making a scene-change based errorconcealment mode decision in a digital video receiver/decoder. At step110 the digital video receiver/decoder verifies the digital videoinformation for the video frame. If no errors are found in the digitalvideo information at step 115, then the digital video receiver/decodercan render the video frame without any help from error concealmentmethods as set forth in step 150.

If errors in the digital video frame information are found, the digitalvideo receiver/decoder then analyzes the digital video frame informationat step 120 to attempt to determine if the damaged video framerepresents a scene-change frame. If the digital video receiver/decoderdetermines that a scene-change has occurred at step 125, then the videoframe is reconstructed with the aid of a spatial error concealmenttechnique as set forth in step 160. Otherwise, when the system hasdetermined that a scene-change has not occurred, the damaged video frameis reconstructed with the help of a temporal error concealment techniqueas set forth in step 170.

One of the most difficult aspects of the error concealment systemdisclosed in FIG. 1 is the analysis of the damaged digital video frameinformation in order to make a decision as to whether a scene-changeoccurred as set forth in steps 120 and 125. Specifically, how can adigital video receiver/decoder accurately determine when a scene-changehas occurred?

One simplistic method of implementing a scene-change error concealmentmode decision is to simply select the error concealment technique basedon the type of video frame that is being decoded. Specifically, a verysimple scene-change based error concealment mode decision method mayselect spatial error concealment (SEC) for all Intra-frames (I-frames)and select temporal error concealment (TEC) for all non Intra-frames(the P-frames and the B-frames). This very simple approach is based uponthe assumption that the digital video encoder encodes an Intra-framewhen ever a scene-change occurs.

A problem with this simplistic scene-change based error concealment modedecision method is that it contains an assumption that is not extremelyaccurate. For example, not all scene changes will be handled with anintra-frame. Furthermore, digital video encoders generally intentionallygenerate intra-frames on a periodic basis even when there is noscene-change. Intra-frames are periodically inserted for the purposes ofrandom access and error recovery. The fact that not all scene-changesare marked with intra-frames and not all intra-frames are scene-changesmakes this very simplistic frame type based scene-change errorconcealment mode decision method sub optimal.

A slightly more sophisticated and effective scene-change detectionmethod detects scene-change by analyzing types of macroblocks used torepresent a digital video frame. In such a scene-change detectionmethod, a frame is determined to be a ‘scene-change’ frame if a largenumber of the macroblocks in the digital video frame are intra-coded.One implementation of this error concealment mode decision may beexpressed with the following equation:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{S\; {EC}},} & {{{if}\mspace{14mu} \frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} \geq {THD}} \\{{TEC},} & {{{if}\mspace{14mu} \frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} < {THD}}\end{matrix} \right.} & (1)\end{matrix}$

wherein

N_(intra,rev)=the number of non-corrupt intra-macroblocks (in a frame)received

N_(total,rev)=the total number of non-corrupt macroblocks (in a frame)received; and

THD=the threshold amount.

Note that N_(total,rev) may not be the total number of macroblocks inthe frame because some macroblocks are missing or corrupt due to packetloss or channel errors.

The scene-change error concealment mode decision method of equation (1)is an improvement over the simplistic frame-type decision method.However, the error concealment mode decision method of equation (1)still has several problems. Four problems with the error concealmentmode decision method of equation (1) are presented in the followingtable.

TABLE 1 Problems with Error Concealment Mode Decision of Equation 1Problem 1. Using the macroblock type as the only indication forscene-change may not be accurate enough. The macroblock type isdetermined by the digital video encoder. Since each different digitalvideo encoder has a different design, one cannot assume that the digitalvideo encoder will act as expected. For example, a digital video encodermay code many macroblocks as “inter” although the residual error aftermotion compensation is significant. Thus, inter blocks may have beenused even when a scene-change has occurred. Problem 2. Since someimportant information (including the header containing macroblock type)may be corrupt or lost, the remaining information may not be sufficientto make a reliable decision. For example, if 97% of macroblocks in aframe are lost and the few remaining macroblocks are all intra-codedthen the error concealment mode decision method of equation (1) willconclude that the frame represents a scene-change such that SEC ischosen. However, the macroblock types of the many lost macroblocks areunknown. It is possible that most of the lost macroblocks were inter-coded such that TEC should have been used. Problem 3. Motion is a factorthat should be taken into more detailed consideration when making anerror concealment mode decision. TEC has a better performance whenconcealing corrupt static scene frames than frames with very highmotion. The simple scene-change detection doesn't take motion intoconsideration. Problem 4. The error concealment mode decision methodspecified in equation (1) does not work well for Intra-frames.Specifically, when all the macroblocks in an Intra-frame areintra-coded, the error concealment mode decision method of equation (1)will always choose SEC for intra- frames. Since Intra-frames areperiodically inserted whether or not a scene-change has actuallyoccurred, SEC will be used even though TEC may have provided betterresults.

An Improved Scene-Change Detector for Error Concealment Mode Decisions

To provide better results in a digital video receiver/decoder, thepresent invention introduces a sophisticated scene-change detectionsystem that may be used for making error concealment mode decisions. Thescene-change detection system of the present invention addresses all ofthe issues presented in Table 1.

Problem 1: Inter-Frames With Large Residual Errors

To address the first problem in Table 1, a factor to reflect the impactof residual errors on inter macroblocks may be added into equation (1).In this manner, a frame with large residual errors from inter-frameencoding may be determined to be a scene-change. In one embodiment, theerror concealment mode decision method of equation (1) is modified tobecome:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{{SE}\; C},} & {{{{if}\mspace{14mu} k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*{Bits}}} \geq T} \\{{TEC},} & {{{{if}\mspace{14mu} k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*{Bits}}} < T}\end{matrix} \right.} & (2)\end{matrix}$

wherein

k₁=a weighting factor for the first term (the % of inter-macroblocks)

Q_(p)=the average quantization parameters of the frame

Bits=the number of bits in the corrupt video frame; and

T=a threshold value.

In the revised error concealment mode decision method of equation (2),the added Q_(p)*Bits adds in the factor of residual errors on intermacroblocks. The idea behind this change is that the residual error foran inter macroblock in a scene-change frame is usually pretty largesince the macroblock cannot find a good match from an earlier referenceframe. Thus, more bits will be needed in order to encode the residualerrors.

In communication networks, digital video information is usuallysegmented into and transmitted by small packets (or cells). The digitalvideo packets may be lost during transmission or discarded by thedigital video receiver/decoder due to a CRC (or checksum) failure.Therefore, the digital video receiver/decoder may not be able to get thecorrupt data such that the “Bits” term (the number of the bits in thecorrupt frame) in equation (2) may be unknown by the digital videodecoder. To handle this, the “Bits” term may be divided into two parts:the number of bits the receiver/decoder receives (denoted asBits_(recv)) and the number of bits that were lost (denoted asBits_(lost)). With this change, equation (2) is modified into:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{S\; {EC}},} & {{{{if}\mspace{14mu} k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {Bits}_{lost}} \right)}} \geq T} \\{{TEC},} & {{{{if}\mspace{14mu} k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {Bits}_{lost}} \right)}} < T}\end{matrix} \right.} & (3)\end{matrix}$

A digital video decoder is generally not informed amount the number ofbits that are lost during transmission. Since the digital video decoderdoes not know the amount of lost data, the Bits_(lost) term should beestimated and provided by the lower layer (e.g. RTP). In one embodiment,the Bits_(lost) term can be estimated using the following equation:

Bits_(lost) =P _(lost) ×S _(lost)   (4)

wherein

P_(lost)=the number of lost packets; and

S_(lost)=the estimated size of those lost packets.

The P_(lost) term can be derived from the sequence numbers in packetheaders. The S_(lost) term can be estimated using the average payloadsize of earlier received digital video packets. Placing equation (4)into equation (3) changes the error concealment mode decision equationinto:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{S\; {EC}},} & {{{{if}\mspace{14mu} k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {P_{lost} \times S_{lost}}} \right)}} \geq T} \\{{TEC},} & {{{{if}\mspace{14mu} k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {P_{lost} \times S_{lost}}} \right)}} < T}\end{matrix} \right.} & (5)\end{matrix}$

Problem 2: Excessive Data Loss

Referring back to Table 1, the second problem with equation (1) is theinaccuracy of the equation if a large amount of digital videoinformation is corrupt or otherwise unavailable. This second problem canbe resolved by adding a scaling “error factor” to the left hand side ofequation that biases the error concealment mode decision method towardtemporal error concealment if a large amount of digital videoinformation is not available. One implementation of adding a scalingfactor to equation (5) is presented below:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{S\; {EC}},} & \begin{matrix}{{if}\mspace{14mu} \left( {1 - {S\left( \frac{N_{corrupt}}{N_{total}} \right)}} \right)} \\{\left( {{k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {P_{lost} \times S_{lost}}} \right)}} \right) \geq T}\end{matrix} \\{{TEC},} & \begin{matrix}{{if}\mspace{14mu} \left( {1 - {S\left( \frac{N_{corrupt}}{N_{total}} \right)}} \right)} \\{\left( {{k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {P_{lost} \times S_{lost}}} \right)}} \right) < T}\end{matrix}\end{matrix} \right.} & (6)\end{matrix}$

wherein

N_(corrupt)=the number of corrupt (including lost) macroblocks in avideo frame;

N_(total)=the total number of macroblocks in the video frame; and

S( )=a bias function.

The bias function S(.) should satisfy S(0)=0 and S(1)=1. In oneembodiment, the S(.) function in equation (6) is a non-linear biasfunction to give different scaling for different corruption rates.Because the number of scene-change frames is generally much less thanthe number of non-scene-change frames, the mode decision is designed tohave a bias in favor of temporal error correction (TEC) as the number ofcorrupt macroblocks increases. Furthermore, spatial error correction(SEC) cannot perform well when most of macroblocks in a digital videoframe are corrupt. In the extreme case, N_(corrupt)=N_(total), the lefthand side of the equation (6) is equal to 0, and temporal errorcorrection (TEC) is always picked.

There are different ways to calculate the number of corrupt macroblockswithin a video frame. The methods will differ depending on whether thedigital video decoder receives the location of errors within a slicefrom a lower layer or not. Each of these two situations is set forthbelow.

If a digital video decoder is informed as where the errors in the videoframe are, an error macroblock map is created such that a macroblock ismarked as “non-corrupt” if no error is found on that macroblock and allprevious macroblocks until previous start code or resync marker.Otherwise, if an error is found in the current macroblock or any earliermacroblock until the previous start code or resync marker, themacroblock is marked as “corrupt”. After the marking, the macroblock mapconsists of error segments wherein each error segment contains aconsecutive number of corrupt macroblocks. An example of a macroblockerror map containing error segments is shown in FIG. 2 wherein shadedareas represent corrupted macroblocks. Note that the term “slice”represents an independent decodable data structure in a video frame.Examples include a “video packet” in the MPEG4 video coding method and a“slice” in the H.263 and AVC video coding methods. The first macroblocknumber in a slice can be obtained from slice header.

If the lower layer does not provide error location information to thedigital video decoder, then the digital video decoder may attempt todetect errors by itself. Generally, such error detection is unreliableand may miss the very first corrupt macroblock. To prevent the problemscaused by missing a corrupt macroblock, a conservative definition of a“non-corrupt macroblock” is used. A macroblock is marked as“non-corrupt” in the macroblock error map if no error is detected in theslice where the macroblock resides. Otherwise, if any error is detectedin the slice then all macroblocks in that slice are marked as “corrupt”.With this conservative definition of corrupt macroblocks, any detectederror in a slice causes the entire slice to be useless. An example ofthis conservative error macroblock map and error segments in this caseis illustrated in FIG. 3.

Problem 3: Motion Level Factor

Referring back to Table 1, the third problem with the scene-changedetection system of equation (1) is that equation (1) fails to take intoconsideration heavy motion. Specifically, when there is a significantamount of motion, temporal error correction becomes less useful.

To address this problem, a “motion level” parameter may be added to theright end of equation (6). This newly added motion level parametercreates an added bias against using temporal error correction when thereis heavy motion in a scene. An embodiment of an error concealmentdecision mode equation with a motion level parameter added to equation(6) is set forth below:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{S\; {EC}},} & \begin{matrix}{{if}\mspace{14mu} \left( {1 - {S\left( \frac{N_{corrupt}}{N_{total}} \right)}} \right)} \\\left( {{k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {P_{lost} \times S_{lost}}} \right)} +} \right. \\{\left. \left\lfloor \frac{\sum\limits_{i,j}\left( {{{{MV}_{x}\left( {i,j} \right)}} + {{{MV}_{y}\left( {i,j} \right)}}} \right)}{k_{2}} \right\rfloor \right) \geq T}\end{matrix} \\{{TEC},} & \begin{matrix}{{if}\mspace{14mu} \left( {1 - {S\left( \frac{N_{corrupt}}{N_{total}} \right)}} \right)} \\\left( {{k_{1}*\frac{N_{{intra},{rcv}}}{N_{{total},{rcv}}}} + {Q_{P}*\left( {{Bits}_{recv} + {P_{lost} \times S_{lost}}} \right)} +} \right. \\{\left. \left\lfloor \frac{\sum\limits_{i,j}\left( {{{{MV}_{x}\left( {i,j} \right)}} + {{{MV}_{y}\left( {i,j} \right)}}} \right)}{k_{2}} \right\rfloor \right) < T}\end{matrix}\end{matrix} \right.} & (7)\end{matrix}$

wherein

MV_(x)(i, j)=the horizontal motion vectors of macroblock (i, j);

MV_(y)(i, j)=the vertical motion vectors of macroblock (i, j); and

k₂=a scaling constant.

Note that the motion vectors for intra-macroblocks and lost macroblocksare not available. Simply assuming zero motion for the missing motionvectors is generally not appropriate. In one embodiment, the “latestavailable” motion vectors are used for corrupt and intra-codedmacroblocks. The “latest available” motion vectors are obtained from“motion vector map” in which the motion information is updated for eachincoming uncorrupt inter-macroblock. Specifically, the motion vector(MV) map consists of M×N entries (M and N are the number of blocks percolumn and per row respectively). Each entry contains the latest motionvectors MV_(x)(i, j) and MV_(y)(i, j) for a block. The motion vector mapmay be initialized with zeros and updated for every non-corrupt interblock that is received.

Problem 4: Intra-Frames

The fourth and final problem listed in Table 1 is that periodicallyinserted intra-frames may trick equation (1) into claiming that ascene-change has occurred when no scene-change has actually occurred.Specifically, an intra-frame that has been interjected into the streamof digital frames for random access and error recovery may cause adigital video receiver/decoder to believe a scene-change has occurred.For resolving this fourth problem, a special scene-change decisionmethod based on similarity instead of macroblock type information isdesigned only for use with I-frames. The bias cost in favor of TEC dueto corruption is added into the decision metric. Similarly, the bias infavor of SEC for motion is also added. The following equation performsscene-change detection and includes a similarity term:

$\begin{matrix}{{Do}\left\{ \begin{matrix}{{S\; {EC}},} & \begin{matrix}{{if}\mspace{14mu} {S\left( \frac{N_{corrupt}}{N_{total}} \right)}\left( {{SIM}_{Y} + {SIM}_{U} + {SIM}_{V} -} \right.} \\{\left. \left\lfloor \frac{\sum\limits_{i,j}\left( {{{{MV}_{x}\left( {i,j} \right)}} + {{{MX}_{y}\left( {i,j} \right)}}} \right)}{C} \right\rfloor \right) < \tau}\end{matrix} \\{{TEC},} & \begin{matrix}{{if}\mspace{14mu} {S\left( \frac{N_{corrupt}}{N_{total}} \right)}\left( {{SIM}_{Y} + {SIM}_{U} + {SIM}_{V} -} \right.} \\{\left. \left\lfloor \frac{\sum\limits_{i,j}\left( {{{{MV}_{x}\left( {i,j} \right)}} + {{{MX}_{y}\left( {i,j} \right)}}} \right)}{C} \right\rfloor \right) \geq \tau}\end{matrix}\end{matrix} \right.} & (8)\end{matrix}$

wherein

C=a scaling constant for motion;

τ=the threshold;

SIM_(Y),=a similarity metric for the luminance domain; and

SIM_(U) and SIM_(V)=similarity metric for the two chrominance domains.

In one embodiment, the similarity metrics may be defined as

$\begin{matrix}{{SIM}_{Y} = \frac{{\overset{\rightarrow}{H}}_{Y,m} \otimes {\overset{\rightarrow}{H}}_{Y,{m - 1}}}{{{\overset{\rightarrow}{H}}_{Y,m}} \otimes {{\overset{\rightarrow}{H}}_{Y,{m - 1}}}}} & (9) \\{{SIM}_{U} = \frac{{\overset{\rightarrow}{H}}_{U,m} \otimes {\overset{\rightarrow}{H}}_{U,{m - 1}}}{{{\overset{\rightarrow}{H}}_{U,m}} \otimes {{\overset{\rightarrow}{H}}_{U,{m - 1}}}}} & (10) \\{{SIM}_{V} = \frac{{\overset{\rightarrow}{H}}_{V,m} \otimes {\overset{\rightarrow}{H}}_{V,{m - 1}}}{{{\overset{\rightarrow}{H}}_{V,m}} \otimes {{\overset{\rightarrow}{H}}_{V,{m - 1}}}}} & (11)\end{matrix}$

where {right arrow over (H)}_(Y,m) and {right arrow over (H)}_(Y,m-1)represent the luminance (Y) histogram for current and previouslyreconstructed frames. The symbols {right arrow over (H)}U,m, {rightarrow over (H)}_(U,m-1), {right arrow over (H)}_(V,m) and {right arrowover (H)}_(V,m-1) for chrominance have similar definitions. In addition,‘{circle around (X)}’ denotes the “inner product”.

FIG. 4 illustrates a revised version of the flow chart of FIG. 1 forperforming scene-change detection for error concealment in a digitalvideo decoder. In the flow diagram of FIG. 4, a decision step 420 hasbeen added to determine if the digital video frame is an Intra-frame(I-frame). If the digital video frame is an Intra-frame then equation(8) will be used for scene-change detection as set forth in step 433.Otherwise, equation (7) will be used for scene-change detection as setforth in step 430.

Improved Spatial Error Concealment

The present invention introduces an improved Spatial Error Concealment(SEC) method for use when spatial error concealment is selected toconceal errors caused by missing or corrupt digital video information.The improved spatial error concealment method of the present inventioncomprises a nearest available pixel based median adaptive predictor withhierarchical interpolation.

The proposed spatial error concealment approach is a macroblock-basedconcealment method and assumes that all correctly received macroblockshave been decoded in advance. In one embodiment, the macroblocks aresixteen by sixteen (16×16) in the luminance domain and eight by eight(8×8) in the chrominance domain as set forth in the well-known ITU andMPEG standards. The spatial error concealment method is performed ondamaged macroblocks in the standard raster order. Thus, all macroblocksand pixels to the left and above the macroblock to be concealed havealready been produced by the decoder and are available for concealingthe damaged macroblock.

The proposed spatial error concealment method divides damagedmacroblocks into four different areas. The proposed spatial errorconcealment method then handles each of the different areas with aspecific spatial error concealment scheme for that area of themacroblock. The four areas and the corresponding pixel coordinates for a16×16 macroblock are defined as:

Area F (lower right corner, 1 pixel): (15, 15)

Area B (bottom row, 15 pixels): (i, 15), 0≦i≦14

Area R (rightmost column, 15 pixels): (15, j), 0≦j≦14

Area M (middle part, 225 pixels): (i, j), 0≦i, j≦14

FIG. 5 graphically illustrates the segmentation of a macroblock into thefour macroblock areas F, B, R, and M.

The proposed spatial error concealment method addresses the fourdifferent macroblock areas in the following order.

1. Reconstruction of area F (the lower right corners): x(15, 15).

2. Reconstruction of area B (the bottom row): x(i, 15), 0≦i≦14.

3. Reconstruction of area R (the rightmost column): x(15, j), 0≦j≦14.

4. Reconstruction of area M (the middle part): x(i, j), 0≦i, j≦14

wherein x(i, j) represents the reconstructed pixel value at pixellocation(i, j). (Note that the reconstruction of area B and area R couldbe performed in reverse order with similar results.) The spatial errorconcealment methods for each of the four different areas are eachdescribed in the following four sections.Reconstruction of Area F (the Lower Right Corner): x(15, 15))

The F area (the lower right corner) is reconstructed by processing thenearest “available” pixels with one of the following three methods:median adaptive prediction, pixel averaging, or pixel copying. A pixelis referred to as “available” if the pixel is located in a non-corruptmacroblock.

Depending on the distance to the nearest available pixels from area F,the reconstruction procedure falls into Case 1 or Case 2 as follows:

Case 1: Case 1 requires that at least one of the adjacent three pixelsin three neighboring macroblocks (below, right, and lower right) beavailable. Specifically, referring to FIG. 6A, one of area F's threeclosest neighbors α, β and γ must be in a non-corrupt macroblocks. Case1 is further divided into the following four sub-cases 1.1 to 1.4:

Case 1.1: Case 1.1 occurs when all of the three surrounding pixels α, βand γ are available (all of the pixels are in non-corrupt macroblocks).When three surrounding pixels α, β and γ are available, then the cornerpixel of area F is reconstructed using the median adaptive predictorwith:

$\begin{matrix}{{x\left( {15,15} \right)} = \left\{ \begin{matrix}{{\min \left( {{x\left( {16,15} \right)},{x\left( {15,16} \right)}} \right)}} & {\begin{matrix}{{{if}\mspace{14mu} {x\left( {16,16} \right)}} \geq {\max\left( {{x\left( {16,15} \right)},} \right.}} \\\left. {x\left( {15,16} \right)} \right)\end{matrix}} \\{{\max \left( {{x\left( {16,15} \right)},{x\left( {15,16} \right)}} \right)}} & {\begin{matrix}{{{if}\mspace{14mu} {x\left( {16,16} \right)}} \leq {\min\left( {{x\left( {16,15} \right)},} \right.}} \\\left. {x\left( {15,16} \right)} \right)\end{matrix}} \\{{{x\left( {16,15} \right)} + {x\left( {15,16} \right)} - {x\left( {16,16} \right)}}} & {{otherwise}}\end{matrix} \right.} & (12)\end{matrix}$

The original median adaptive predictor, designed to compress imageslosslessly, is a nonlinear predictor to predict a current pixel x bythree surrounding pixels p, q and r as illustrated in FIG. 6B. Itassumes that a vertical edges lies above x if r≧q ≧p or r≦q≦p, and ahorizontal edge lies above x if r≧p≧q or r≦p≦q. Note that the threesurrounding pixels used to reconstruct area F are different from thosein the original median adaptive predictor.

Case 1.2: Case 1.2 is used when both pixels α and β of FIG. 6A are innon-corrupt macroblocks (but γ is not available since that would be case1.1). In case 1.2, the corner pixel of area F is reconstructed usingpixel averaging with the following equation:

$\begin{matrix}{{x\left( {15,15} \right)} = \frac{\left( {{x\left( {16,15} \right)} + {x\left( {15,16} \right)} + 1} \right)}{2}} & (13)\end{matrix}$

wherein the divide by two operation may be performed with a shift rightoperation (>>1).

Case 1.3: Case 1.3 is used when either α or β (but not both) is in anon-corrupt macroblock (the status of γ is irrelevant). In case 1.3,area F is reconstructed with pixel copying as:

x(15,15)=x(16,15) or x(15,16)   (14)

Case 1.4: Finally, Case 1.4 is used when only γ is in a non-corruptmacroblock. In case 1.4, area F is reconstructed with pixel copying as:

x(15,15)=x(16,16)   (15)

Case 2: Referring back to FIG. 6A, Case 2 occurs when area F's threeclosest neighbors α, β and γ are all in corrupt macroblocks. Since α, βand γ of FIG. 6A are not available, area F must be reconstructed usingpixels from the other three macroblocks (above, left and upper left)adjacent to the macroblock currently being concealed. Note that thesemacroblocks may not be available if the current macroblock is located onan edge. If the macroblock is not located on an edge, these macroblockshave already been reconstructed since they were received without erroror already concealed as illustrated in FIG. 5. In case 2, thebottom-rightmost pixel of the neighboring macroblocks will be usedreconstruct area F as illustrated by pixels α, β and γ in FIG. 7. Due tothe edge situations, case 2 is further divided into the following foursub-cases 2.1 to 2.4,

Case 2.1: Case 2.1 occurs when macroblock is located in the upper leftcorner of a frame. In this case, no nearby pixel information wasavailable so area F is just set to an average or median pixel value. Inone embodiment, area F is reconstructed with the following equation:

x(15,15)=128   (16)

Case 2.2: Case 2.2 occurs when the macroblock is located in the top rowof a frame. In Case 2.2, area F is reconstructed by pixel copying α inFIG. 7 with the following equation:

x(15,15)=x(−1,15)   (17)

Case 2.3: Case 2.3 occurs when the macroblock is located in the leftmostcolumn of the frame. In case 2.3, area F is reconstructed by pixelcopying β of FIG. 7 with the following equation:

x(15,15)=x(15,−1)   (18)

Case 2-4: Case 2.1 occurs when none of the above cases apply (all of theupper and left macroblocks are available). In case 2.4, area F isreconstructed by applying the median adaptive predictor on x(−1, 15),x(15, −1), and x(−1, −1) (pixels α, β and γ as illustrated in FIG. 7)with the following equation:

$\begin{matrix}{{x\left( {15,15} \right)} = \left\{ \begin{matrix}{{\min \left( {{x\left( {{- 1},15} \right)},{x\left( {15,{- 1}} \right)}} \right)}} & {\begin{matrix}{{{if}\mspace{14mu} {x\left( {{- 1},{- 1}} \right)}} \geq \max} \\\left( {{x\left( {{- 1},15} \right)},{x\left( {15,{- 1}} \right)}} \right.\end{matrix}} \\{{\max \left( {{x\left( {{- 1},15} \right)},{x\left( {15,{- 1}} \right)}} \right)}} & {\begin{matrix}{{{if}\mspace{14mu} {x\left( {{- 1},{- 1}} \right)}} \leq \min} \\\left( {{x\left( {{- 1},15} \right)},{x\left( {15,{- 1}} \right)}} \right.\end{matrix}} \\{{{x\left( {{- 1},15} \right)} + {x\left( {15,{- 1}} \right)} - {x\left( {{- 1},{- 1}} \right)}}} & {{otherwise}}\end{matrix} \right.} & (19)\end{matrix}$

Reconstruction of Area B (the Bottom Row): x(i, 15), 0≦i≦14

Referring back to FIG. 6A, two different reconstruction schemes are usedto recover the bottom row area B. Specifically, Case 1 or Case 2 setforth below will be selected depending on whether the macroblock belowrow area B is available or corrupt or otherwise not available.

Case 1: Case 1 is used when the macroblock below row area B isnon-corrupt. In this case, each pixel in the area B is reconstructed asthe same value as the pixel below it using the following equation:

x(i,15)=x(i,16), 0≦i≦14   (20)

Case 2: Case 2 is used when the macroblock below row area B is corruptor not available. In this case, each of the pixels in the row area B isreconstructed as the average of two previously reconstructed pixelsstarting from the middle of the bottom row. The initial two previouslyreconstructed pixels are the pixel from area F and the bottom-rightmostpixel in the macroblock to the left. The reconstruction of the pixels isperformed using the following equation:

$\begin{matrix}{{x\left( {i,15} \right)} = \frac{\left( {{x\left( {{i - k},15} \right)} + {x\left( {{i + k},15} \right)} + 1} \right)}{2}} & (21)\end{matrix}$

wherein the reconstruction order and the relationship between i and kare set forth in the following Table 2. Note that the divide-by-twooperation can be performed with a shift right operation (>>1).

TABLE 2 Reconstruction order and the relationship between i and k forareas B & R Order i k 1 7 8 2 3 4 3 11 4 4 1 2 5 5 2 6 9 2 7 13 2 8 0 19 2 1 10 4 1 11 6 1 12 8 1 13 10 1 14 12 1 15 14 1

The reconstruction procedure is graphically illustrated in FIGS. 8A to8C. Each pixel in the bottom row, x(i, 15), is reconstructed by theaverage of the two closest previous reconstructed pixels. Each pixelrequires two add operations (+) and one shift operation (>>) for thereconstruction.

Reconstruction of R (the Rightmost Column): x(15, i), 0≦i≦14

The reconstruction of the rightmost column R area in FIG. 6A proceeds inthe same manner as the reconstruction procedure for the bottom row areaB. Specifically, two different schemes are used to conceal the rightmostcolumn R area of a damaged macroblock depending on whether or not themacroblock to the right of column area R is corrupt.

Case 1: Case 1 is used when the macroblock to the right of the columnarea R is non-corrupt. In this case, each pixel in the area R isreconstructed the same value as the pixel next to it with the followingequation:

x(15,i)=x(16,i), 0≦i≦14   (22)

Case 2: Case 2 is used when the macroblock to the right of the columnarea R is corrupt or not available. In case 2, each pixel in the area Ris reconstructed as the average of two previously reconstructed pixelsstarting from the middle of the rightmost column with the followingequation:

$\begin{matrix}{{x\left( {15,i} \right)} = \frac{\left( {{x\left( {15,{i - k}} \right)} + {x\left( {15,{i + k}} \right)} + 1} \right)}{2}} & (23)\end{matrix}$

The reconstruction order and the relationship between i and k are thesame as those in the previous section on reconstruction row area B. Notethat the divide-by-two operation can be performed with a shift rightoperation (>>1).Reconstruction of Area M (the Middle Part): x(i, j), 0≦i, i≦14

At this point, the pixels in the bottom row and the rightmost column ofthe macroblock have been reconstructed. The remaining pixels (the middlepart of the macroblock, area M as illustrated in FIG. 6A) arereconstructed as the average of four previously reproduced pixelsstarting from the middle of the macroblock. Specifically, the followingequation can be used to fill in the pixels of area M:

$\begin{matrix}{{x\left( {i,j} \right)} = \frac{\begin{matrix}\left( {{x\left( {{i - m},{j - n}} \right)} + {x\left( {{i + m},{j + n}} \right)} +} \right. \\\left. {{x\left( {{i + n},{j - m}} \right)} + {x\left( {{i - n},{j + m}} \right)} + 2} \right)\end{matrix}}{4}} & (24)\end{matrix}$

wherein n=0 or m. Note that the division by four in equation (24) can beperformed with a right shift two places operation (>>2). Thereconstruction order and the relationship between (i, j), m and n forthe first 25 pixels are shown in Table 3. The order of the remainingpixels can be derived accordingly.

TABLE 3 Reconstruction order and the relationship between i and k forarea M. order (i, j) m n 1 (7, 7) 8 0 2 (3, 3) 4 4 3 (11, 3) 4 4 4 (3,11) 4 4 5 (11, 11) 4 4 6 (7, 3) 4 0 7 (3, 7) 4 0 8 (11, 7) 4 0 9 (7, 11)4 0 10 (1, 1) 2 2 11 (5, 1) 2 2 12 (9, 1) 2 2 13 (13, 1) 2 2 14 (1, 5) 22 15 (5, 5) 2 2 16 (9, 5) 2 2 17 (13, 5) 2 2 18 (1, 9) 2 2 19 (5, 9) 2 220 (9, 9) 2 2 21 (13, 9) 2 2 22 (1, 13) 2 2 23 (5, 13) 2 2 24 (9, 13) 22 25 (13, 13) 2 2 etc.Each pixel reconstruction takes four add operations (+) and one shiftright two places (>>2) operation. The reconstruction procedure for AreaM is graphically illustrated in FIGS. 9A and 9B.

Errors Propagated to Uncorrupted Frames

Even when the information for a digital video frame is transmitted andreceived without any error, the displayed digital video frame may bevisually degraded due to errors in earlier received digital videoframes. Specifically, a current digital video frame may containreferences to corrupted macroblocks in earlier transmitted digital videoframes. If those corrupted macroblocks were reconstructed with an errorconcealment method, the current digital video frame that referencesthose reconstructed macroblocks will contain visual artifacts.

To address this problem, the present invention proposes “refreshing”macroblocks under certain conditions. The term “refreshing” meansreproducing macroblocks by spatial error concealment (SEC). To preventover-shooting (e.g. blurring) introduced by the SEC, the “refreshing”scheme is only applied on the chrominance (U and V) domain.

The refreshing procedure of the present invention includes the followingthree steps: preparation, decision and processing. Each step will bedescribed in detail.

Preparation: A Refreshing Table

The system creates a refreshing table consisting of M×N entries whereineach entry represents the “refreshing” status for a macroblock (M and Ndenote the number of rows and columns of macroblocks in a digitalframe). The system sets the initial value of an entry as ‘2’ if thecorresponding macroblock is corrupt and concealed by spatial errorcorrection. Otherwise, set the initial value as ‘0’.

Decision: Use Spatial Error Concealment for Non Corrupt Block?

Next, the system scans the digital video frame and changes therefreshing table entry value from ‘0’ to ‘1’ if the correspondingmacroblock satisfies all the following conditions:

-   -   1. It is a non corrupt macroblock (If the macroblock is corrupt        then error concealment will be triggered.)    -   2. It is an inter-macroblock (intra macroblocks have no error        propagation problem)    -   3. The number of non-corrupt macroblocks is greater than a        threshold. (SEC cannot perform well in the condition of too many        corrupt macroblocks.)        Otherwise, the system leaves the value as it is.

Because the purpose of the proposed algorithm is to reduce the impact oferror propagation on non corrupt inter-macroblocks, the decision shouldbe made based on the error conditions. Accordingly, the threshold incondition 3 should be set as a function of damage. The threshold shouldbe set lower such that more macroblocks will be refreshed when damage ismore significant. To reflect the true impact, the damage is calculatednot only by the number of corrupt macroblocks, but also by the otherfactors, such as the amount of motion, whether a scene-change occurred,non-corrupt intra-macroblocks, past damage, etc. Factors such as largeamounts of motion and a scene-change occurring tend to make the damageeven worse. On the other hand, non-corrupt intra-macroblocks tend toreduce the damage. In one embodiment, the damage factor is calculatedwith the following equation:

D _(i) =k ₁(|MV _(X) |+|MV _(Y)|)×(sN _(i) +k ₂(|MV_(X) |+|MV _(Y)|)ID_(i-1))

wherein:

-   -   |MV_(X) 51 +|MV_(Y)|=motion vector magnitude    -   N_(i)=new damage (number of corrupt macroblocks)    -   D_(i-1)=past damage    -   k₁, k₂=constants to reflect damage expansion caused by motion        (k₁, k₂≧1). k₁=k₂=1 for static scene.    -   I=error correction factor (a benefit gained from non-corrupt        Intra-macroblocks). I≦1 (1 indicating no non-corrupt        Intra-macroblocks)    -   s=scene-change factor, s>1 (1 indicating no scene-change        occurred)

Processing: Perform the Spatial Error Concealment

Apply the spatial error concealment method on the chrominance domain fora macroblock whose corresponding refreshing table entry is marked as‘1’. Apply the spatial error concealment algorithm on both the luminancedomain and chrominance domain for each macroblock marked as ‘2’. Manydifferent spatial error concealment methods may be used. In oneembodiment, the spatial error concealment method based set forth in theprevious section serves as spatial error concealment for “refreshing” inthe experiment.

The foregoing has described a number of techniques for performing errorconcealment in digital video. It is contemplated that changes andmodifications may be made by one of ordinary skill in the art, to thematerials and arrangements of elements of the present invention withoutdeparting from the scope of the invention.

1. A method of determining if a digital video frame represents ascene-change in a digital video stream, said method comprising:determining if said digital video frame is an intra-frame, and applyingusing a first equation comprising a similarity metric to determine if ascene-change occurred if said digital video frame is an intra-frame;applying a second equation comprising an intra-macroblock ratio term todetermine if a scene-change occurred if said digital video frame is notan intra-frame.
 2. The method of determining if a digital video framerepresents a scene-change in a digital video stream as set forth inclaim 1 wherein said second equation comprises a frame bit size term. 3.The method of determining if a digital video frame represents ascene-change in a digital video stream as set forth in claim 2 whereinsaid frame bit size term comprises a quantization parameter multipliedby a number of bits for said digital video frame.
 4. The method ofdetermining if a digital video frame represents a scene-change in adigital video stream as set forth in claim 3 wherein said number of bitsfor said frame comprises a number of bits received for said digitalvideo frame plus a number of packets lost times an average packet size.5. The method of determining if a digital video frame represents ascene-change in a digital video stream as set forth in claim 1 whereinsaid second equation further comprises a motion level term thatspecifies an amount of motion.
 6. The method of determining if a digitalvideo frame represents a scene-change in a digital video stream as setforth in claim 1 wherein said second equation further comprises a dataloss scaling factor.
 7. The method of determining if a digital videoframe represents a scene-change in a digital video stream as set forthin claim 6 wherein said data loss scaling factor biases towards temporalerror concealment.
 8. A method of determining if a digital video framerepresents a scene-change in a digital video stream, said methodcomprising: resolving a multi-term equation to determine if ascene-change occurred, said multi-term equation comprising anintra-macroblock ratio term.
 9. The method of determining if a digitalvideo frame represents a scene-change in a digital video stream as setforth in claim 8 wherein said multi-term equation further comprises aframe size term.
 10. The method of determining if a digital video framerepresents a scene-change in a digital video stream as set forth inclaim 9 wherein said frame size term comprises a quantization parametermultiplied by a number of bits for said digital video frame.
 11. Themethod of determining if a digital video frame represents a scene-changein a digital video stream as set forth in claim 10 wherein said numberof bits for said frame comprises a number of bits received for saiddigital video frame plus a number of packets lost times an averagepacket size.
 12. The method of determining if a digital video framerepresents a scene-change in a digital video stream as set forth inclaim 8 wherein said multi-term equation further comprises a motionlevel term that specifies an amount of motion.
 13. The method ofdetermining if a digital video frame represents a scene-change in adigital video stream as set forth in claim 8 wherein said multi-termequation further comprises a data loss scaling factor.
 14. The method ofdetermining if a digital video frame represents a scene-change in adigital video stream as set forth in claim 13 wherein said data lossscaling factor biases towards temporal error concealment.
 15. A computerreadable medium, said computer readable medium comprising a series ofinstructions for determining if a digital video frame represents ascene-change in a digital video stream, said series of instructionsimplementing the steps of: resolving a multi-term equation to determineif a scene-change occurred, said multi-term equation comprising anintra-macroblock ratio term.
 16. The computer readable medium as setforth in claim 15 wherein said multi-term equation further comprises aframe size term.
 17. The computer readable medium as set forth in claim16 wherein said frame size term comprises a quantization parametermultiplied by a number of bits for said digital video frame.
 18. Thecomputer readable medium as set forth in claim 17 wherein said number ofbits for said frame comprises a number of bits received for said digitalvideo frame plus a number of packets lost times an average packet size.19. The computer readable medium as set forth in claim 15 wherein saidmulti-term equation further comprises a motion level term that specifiesan amount of motion.
 20. The computer readable medium as set forth inclaim 15 wherein said multi-term equation further comprises a data lossscaling factor.
 21. The computer readable medium as set forth in claim20 wherein said data loss scaling factor biases towards temporal errorconcealment.
 22. A digital video decoder, said digital video decodermethod of determining if a digital video frame represents a scene-changein a digital video stream, said method comprising: means for resolving amulti-termed equation to determine if a scene-change occurred, saidmulti-termed equation comprising an intra-macroblock ratio term; meansfor performing spatial error concealment if said multi-termed equationdetermined that a scene-change occurred; and means for performingtemporal error concealment if said multi-termed equation determined thata scene-change did not occur.
 23. A method of performing spatial errorconcealment, said method comprising the steps of: dividing a corruptmacroblock into more than one region, a first region of said corruptmacroblock comprising a corner pixel of said macroblock; determining apixel value for said first region comprising said corner pixel; anddetermining pixel values for said regions other than said first regionusing said pixel value for said corner pixel.
 24. The method ofperforming spatial error concealment as set forth in claim 23 whereindetermining a pixel value for said first region comprises interpolatingsaid first region using adjacent pixels in adjacent macroblocks.
 25. Themethod of performing spatial error concealment as set forth in claim 23wherein determining a pixel value for said first region comprises anadjacent pixel in an adjacent macroblock.
 26. The method of performingspatial error concealment as set forth in claim 23 wherein said morethan one region further comprise a pixel row adjacent to said cornerpixel and a pixel column adjacent to said corner pixel.
 27. The methodof performing spatial error concealment as set forth in claim 26 whereinsaid pixel row region is constructed with a pixel row from an adjacentmacroblock if available else reconstructed using said corner region. 28.The method of performing spatial error concealment as set forth in claim26 wherein said pixel row region is constructed with a pixel column froman adjacent macroblock if available else reconstructed using said cornerregion.
 29. The method of performing spatial error concealment as setforth in claim 26 wherein said more than one region further comprisemain region bordered by said pixel row adjacent to said corner pixel,said pixel column adjacent to said corner pixel, and said corner pixel.30. The method of performing spatial error concealment as set forth inclaim 29 wherein said main region is reconstructed by interpolatingbetween adjacent macroblocks and said pixel row adjacent to said cornerpixel, said pixel column adjacent to said corner pixel, and said cornerpixel.
 31. A computer readable medium, said computer readable mediumcomprising a set of instructions for performing spatial errorconcealment, said instructions performing the steps of: dividing acorrupt macroblock into more than one region, a first region of saidcorrupt macroblock comprising a corner pixel of said macroblock;determining a pixel value for said first region comprising said cornerpixel; and determining pixel values for said regions other than saidfirst region using said pixel value for said corner pixel.
 32. Thecomputer readable medium as set forth in claim 31 wherein determining apixel value for said first region comprises interpolating said firstregion using adjacent pixels in adjacent macroblocks.
 33. The computerreadable medium as set forth in claim 31 wherein said more than oneregion further comprise a pixel row adjacent to said corner pixel and apixel column adjacent to said corner pixel.
 34. The computer readablemedium as set forth in claim 33 wherein said pixel row region isconstructed with a pixel row from an adjacent macroblock if availableelse reconstructed using said corner region.
 35. The computer readablemedium as set forth in claim 33 wherein said pixel row region isconstructed with a pixel column from an adjacent macroblock if availableelse reconstructed using said corner region.
 36. The computer readablemedium as set forth in claim 33 wherein said more than one regionfurther comprise main region bordered by said pixel row adjacent to saidcorner pixel, said pixel column adjacent to said corner pixel, and saidcorner pixel.
 37. The computer readable medium as set forth in claim 36wherein said main region is reconstructed by interpolating betweenadjacent macroblocks and said pixel row adjacent to said corner pixel,said pixel column adjacent to said corner pixel, and said corner pixel.38. A spatial error concealment apparatus, said spatial error apparatuscomprising: means for dividing a corrupt macroblock into more than oneregion, a first region of said corrupt macroblock comprising a cornerpixel of said macroblock; means for determining a pixel value for saidfirst region comprising said corner pixel; and means for determiningpixel values for said regions other than said first region using saidpixel value for said corner pixel.
 39. The spatial error concealmentapparatus as set forth in claim 31 wherein means for determining a pixelvalue for said first region comprising said corner pixel comprisesinterpolating said first region using adjacent pixels in adjacentmacroblocks.
 40. A method of refreshing macroblocks when performingerror concealment, said method comprising the steps of: determining if amacroblock should be refreshed by determining if said macroblock is notcorrupt; determining if said macroblock is an inter-macroblock;determining if a damage factor has exceeded a threshold; and refreshingsaid macroblock if said macroblock needs to be refreshed by usingspatial error concealment on said macroblock.
 41. The method ofrefreshing macroblocks as set forth in claim 40 wherein said refreshingis only performed in a chrominance domain.
 42. The method of refreshingmacroblocks as set forth in claim 40 wherein said damage factor isincreased by a number of corrupt macroblocks.
 43. The method ofrefreshing macroblocks as set forth in claim 40 wherein said damagefactor is increased by an amount of motion.
 44. The method of refreshingmacroblocks as set forth in claim 40 wherein said damage factor isdecreased by an amount of intra-macroblocks.
 45. The method ofrefreshing macroblocks as set forth in claim 40 wherein said damagefactor is increased when a scene-change is detected.
 46. A computerreadable medium, said computer readable medium comprising set ofinstructions for refreshing macroblocks when performing errorconcealment, said instructions implementing the steps of: determining ifa macroblock should be refreshed by determining if said macroblock isnot corrupt; determining if said macroblock is an inter-macroblock;determining if a damage factor has exceeded a threshold; and refreshingsaid macroblock if said macroblock needs to be refreshed by usingspatial error concealment on said macroblock.
 47. The computer readablemedium as set forth in claim 46 wherein said refreshing is onlyperformed in a chrominance domain.
 48. The computer readable medium asset forth in claim 46 wherein said damage factor is increased by anumber of corrupt macroblocks.
 49. The computer readable medium as setforth in claim 46 wherein said damage factor is increased by an amountof motion.
 50. The computer readable medium as set forth in claim 46wherein said damage factor is decreased by an amount ofintra-macroblocks.
 51. The computer readable medium as set forth inclaim 46 wherein said damage factor is increased when a scene-change isdetected.